Non-pharmaceutical interventions during COVID-19 in the UK and Spain: a rapid realist review.

The paper is located at the crossroads of two modern intellectual movements. The first, evidence-based policy, seeks to locate vital information that will inform and improve key policy decisions on such matters as population health, social welfare, and human wellbeing. The second, complexity theory, describes the nature of the social world and perceives human action as persistently adaptive and social institutions as incessantly self-transformative. The first assumes that policies and programmes can achieve sufficient control to meet specific and measurable objectives. The second assumes that social actions are sufficiently capricious so that the society never conforms to anyone's plans - even those of the most powerful. The unparalleled resources committed to control the unprecedented attack of the COVID-19 pandemic are the epitome of complexity. The long struggle to contain the virus thus constitutes an ideal test bed to investigate this paradigmatic split. The paper undertakes this mission - focusing specifically on the effectiveness non-pharmaceutical interventions and examining evidence from the UK and Spain.

Pragmatic trials and implementation science: grounds for divorce?

BACKGROUND: The paper opens with a brief history of two of the major intellectual components of the recent utilitarian turn in clinical research, namely 'pragmatic trials' and 'implementation science'. The two schools of thought developed independently and the paper scrutinises their mutual compatibilities and incompatibilities, asking: i) what do the leading advocates of pragmatic trials assume about the transfer of research findings to real-world practice and ii) what role pragmatic trials can and should play in the evaluation of implementation science strategies. METHODS: The paper utilises 'explication de texte': i) providing a close reading of the inferential logics contained in major published expositions of the two paradigms, and ii) interrogating the conclusions of a pragmatic trial of an intervention providing guidelines on retinal screening aimed at family practitioners. RESULTS: The paper is in two parts. Part 1 unearths some significant incommensurability - the pragmatic trial literature retains an antiquated view of knowledge transfer and is overly optimistic about the wide applicability the findings of pragmatic trials to 'real world' conditions. Part 2 of the paper outlines an empirical strategy to better penetrate the mechanisms of knowledge transfer and to tackle the issue of the generalisabilty of research findings in implementation science. CONCLUSIONS: Pragmatism, classically, is about problem solving and the melding of perspectives. The core research requirement in implementation science is a fundamental shift from the narrow shoulders of pragmatic trials to a model of explanation building based upon a multi-case, multi-method body of evidence.

The "pragmatic trial": An essentially contested concept?

For over 50 years, clinical research methodology has wrestled with the problem of the lack of correspondence between tests of treatments and applications of treatments. The former comprise of trials featuring scrupulous control of patient eligibility, treatment compliance, clinician expertise, follow-up intensity, and so on. In applying a validated treatment, the practitioner has to confront considerable real-world variation in potential patients and in implementation regimes. The remedy, going by the name of "pragmatic trials," is to conduct clinical trials in conditions corresponding more closely to everyday practice. This solution has proved easier to utter than to execute, and the paper reviews the extensive literature on pragmatic trials, seeking to assess whether it has terminated in clarity or contestation.

Guidance on guidelines: Understanding the evidence on the uptake of health care guidelines.

RATIONALE: Regardless of health issue, health sector, patient condition, or treatment modality, the chances are that provision is supported by "a guideline" making professionally endorsed recommendations on best practice. Against this background, research has proliferated seeking to evaluate how effectively such guidance is followed. These investigations paint a gloomy picture with many a guideline prompting lip service, inattention, and even opposition. This predicament has prompted a further literature on how to improve the uptake of guidelines, and this paper considers how to draw together lessons from these inquiries. METHODS: This huge body of material presents a considerable challenge for research synthesis, and this paper produces a critical, methodological comparison of 2 types of review attempting to meet that task. Firstly, it provides an overview of the current orthodoxy, namely, "thematic reviews," which aggregate and enumerate the "barriers and facilitators" to guideline implementation. It then outlines a "realist synthesis," focussing on testing the "programme theories" that practitioners have devised to improve guideline uptake. RESULTS: Thematic reviews aim to provide a definitive, comprehensive catalogue of the facilitators and barriers to guideline implementation. As such, they present a restatement of the underlying problems rather than an improvement strategy. The realist approach assumes that the incorporation of any guideline into current practice will produce unintended system strains as different stakeholders wrestle over responsibilities. These distortions will prompt supplementary revisions to guidelines, which in turn beget further strains. Realist reviews follow this dynamic understanding of organisational change. CONCLUSIONS: Health care decision makers operate in systems that are awash with guidelines. But guidelines only have paper authority. Managers do not need a checklist of their pros and cons, because the fate of guidelines depends on their reception rather than their production. They do need decision support on how to engineer and reengineer guidelines so they dovetail with evolving systems of health care delivery.

Protocol--the RAMESES II study: developing guidance and reporting standards for realist evaluation.

INTRODUCTION: Realist evaluation is an increasingly popular methodology in health services research. For realist evaluations (RE) this project aims to: develop quality and reporting standards and training materials; build capacity for undertaking and critically evaluating them; produce resources and training materials for lay participants, and those seeking to involve them. METHODS: To achieve our aims, we will: (1) Establish management and governance infrastructure; (2) Recruit an interdisciplinary Delphi panel of 35 participants with diverse relevant experience of RE; (3) Summarise current literature and expert opinion on best practice in RE; (4) Run an online Delphi panel to generate and refine items for quality and reporting standards; (5) Capture 'real world' experiences and challenges of RE-for example, by providing ongoing support to realist evaluations, hosting the RAMESES JISCmail list on realist research, and feeding problems and insights from these into the deliberations of the Delphi panel; (6) Produce quality and reporting standards; (7) Collate examples of the learning and training needs of researchers, students, reviewers and lay members in relation to RE; (8) Develop, deliver and evaluate training materials for RE and deliver training workshops; and (9) Develop and evaluate information and resources for patients and other lay participants in RE (eg, draft template information sheets and model consent forms) and; (10) Disseminate training materials and other resources.Planned outputs: (1) Quality and reporting standards and training materials for RE. (2) Methodological support for RE. (3) Increase in capacity to support and evaluate RE. (4) Accessible, plain-English resources for patients and the public participating in RE. DISCUSSION: The realist evaluation is a relatively new approach to evaluation and its overall place in the is not yet fully established. As with all primary research approaches, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency.

Providing effective and preferred care closer to home: a realist review of intermediate care.

Intermediate care is one of the number of service delivery models intended to integrate care and provide enhanced health and social care services closer to home, especially to reduce reliance on acute care hospital beds. In order for health and social care practitioners, service managers and commissioners to make informed decisions, it is vital to understand how to implement the admission avoidance and early supported discharge components of intermediate care within the context of local care systems. This paper reports the findings of a theory-driven (realist) review conducted in 2011-2012. A broad range of evidence contained in 193 sources was used to construct a conceptual framework for intermediate care. This framework forms the basis for exploring factors at service user, professional and organisational levels that should be considered when designing and delivering intermediate care services within a particular local context. Our synthesis found that involving service users and their carers in collaborative decision-making about the objectives of care and the place of care is central to achieving the aims of intermediate care. This pivotal involvement of the service user relies on practitioners, service managers and commissioners being aware of the impact that organisational structures at the local level can have on enabling or inhibiting collaborative decision-making and care co-ordination. Through all interactions with service users and their care networks, health and social care professionals should establish the meaning which alternative care environments have for different service users. Doing so means decisions about the best place of care will be better informed and gives service users choice. This in turn is likely to support psychological and social stability, and the attainment of functional goals. At an organisational level, integrated working can facilitate the delivery of intermediate care, but there is not a straightforward relationship between integrated organisational processes and integrated professional practice.

Exploring complex causal pathways between urban renewal, health and health inequality using a theory-driven realist approach.

INTRODUCTION: Urban populations are growing and to accommodate these numbers, cities are becoming more involved in urban renewal programs to improve the physical, social and economic conditions in different areas. This paper explores some of the complexities surrounding the link between urban renewal, health and health inequalities using a theory-driven approach. METHODS: We focus on an urban renewal initiative implemented in Barcelona, the Neighbourhoods Law, targeting Barcelona's (Spain) most deprived neighbourhoods. We present evidence from two studies on the health evaluation of the Neighbourhoods Law, while drawing from recent urban renewal literature, to follow a four-step process to develop a program theory. We then use two specific urban renewal interventions, the construction of a large central plaza and the repair of streets and sidewalks, to further examine this link. DISCUSSION: In order for urban renewal programs to affect health and health inequality, neighbours must use and adapt to the changes produced by the intervention. However, there exist barriers that can result in negative outcomes including factors such as accessibility, safety and security. CONCLUSION: This paper provides a different perspective to the field that is largely dominated by traditional quantitative studies that are not always able to address the complexities such interventions provide. Furthermore, the framework and discussions serve as a guide for future research, policy development and evaluation.

Evaluating deceased organ donation: a programme theory approach.

PURPOSE: Organ donation and transplantation services represent a microcosm of modern healthcare organisations. They are complex adaptive systems. They face perpetual problems of matching supply and demand. They operate under fierce time and resource constraints. And yet they have received relatively little attention from a systems perspective. The purpose of this paper is to consider some of the fundamental issues in evaluating, improving and policy reform in such complex systems. DESIGN/METHODOLOGY/APPROACH: The paper advocates an approach based on programme theory evaluation. FINDINGS: The paper explains how the death to donation to transplantation process depends on the accumulation of series of embedded, institutional sub-processes. Evaluators need to be concerned with this whole system rather than with its discrete parts or sectors. Policy makers may expect disappointment if they seek to improve donation rates by applying nudges or administrative reforms at a single point in the implementation chain. ORIGINALITY/VALUE: These services represent concentrated, perfect storms of complexity and the paper offers guidance to practitioners with bio-medical backgrounds on how such services might be evaluated and improved. For the methodological audience the paper caters for the burgeoning interest in programme theory evaluation while illustrating the design phase of this research strategy.

Functionality and feedback: a protocol for a realist synthesis of the collation, interpretation and utilisation of PROMs data to improve patient care.

INTRODUCTION: The feedback and public reporting of PROMs data aims to improve the quality of care provided to patients. Existing systematic reviews have found it difficult to draw overall conclusions about the effectiveness of PROMs feedback. We aim to execute a realist synthesis of the evidence to understand by what means and in what circumstances the feedback of PROMs data leads to the intended service improvements. METHODS AND ANALYSIS: Realist synthesis involves (stage 1) identifying the ideas, assumptions or 'programme theories' which explain how PROMs feedback is supposed to work and in what circumstances and then (stage 2) reviewing the evidence to determine the extent to which these expectations are met in practice. For stage 1, six provisional 'functions' of PROMs feedback have been identified to structure our review (screening, monitoring, patient involvement, demand management, quality improvement and patient choice). For each function, we will identify the different programme theories that underlie these different goals and develop a logical map of the respective implementation processes. In stage 2, we will identify studies that will provide empirical tests of each component of the programme theories to evaluate the circumstances in which the potential obstacles can be overcome and whether and how the unintended consequences of PROMs feedback arise. We will synthesise this evidence to (1) identify the implementation processes which support or constrain the successful collation, interpretation and utilisation of PROMs data; (2) identify the implementation processes through which the unintended consequences of PROMs data arise and those where they can be avoided. ETHICS AND DISSEMINATION: The study will not require NHS ethics approval. We have secured ethical approval for the study from the University of Leeds (LTSSP-019). We will disseminate the findings of the review through a briefing paper and dissemination event for National Health Service stakeholders, conferences and peer reviewed publications.

Do reviews of healthcare interventions teach us how to improve healthcare systems?

Planners, managers and policy makers in modern health services are not without ingenuity - they will always try, try and try again. They face deep-seated or 'wicked' problems, which have complex roots in the labyrinthine structures though which healthcare is delivered. Accordingly, the interventions devised to deal with such stubborn problems usually come in the plural. Many different reforms are devised to deal with a particular stumbling block, which may be implemented sequentially, simultaneously or whenever policy fashion or funding dictates. This paper examines this predicament from the perspective of evidence based policy. How might researchers go about reviewing the evidence when they are faced with multiple or indeed competing interventions addressing the same problem? In the face of this plight a rather unheralded form of research synthesis has emerged, namely the 'typological review'. We critically review the fortunes of this strategy. Separating the putative reforms into series of subtypes and producing a scorecard of their outcomes has the unintended effect of divorcing them all from an understanding of how organisations change. A more fruitful approach may lie in a 'theory-driven review' underpinned by an understanding of dynamics of social change in complex organisations. We test this thesis by examining the primary and secondary research on the many interventions designed to tackle a particularly wicked problem, namely the inexorable rise in demand for healthcare.

Realist RCTs of complex interventions - an oxymoron.

Bonell et al. discuss the challenges of carrying out randomised controlled trials (RCTs) to evaluate complex interventions in public health, and consider the role of realist evaluation in enhancing this design (Bonell, Fletcher, Morton, Lorenc, & Moore, 2012). They argue for a "synergistic, rather than oppositional relationship between realist and randomised evaluation" and that "it is possible to benefit from the insights provided by realist evaluation without relinquishing the RCT as the best means of examining intervention causality." We present counter-arguments to their analysis of realist evaluation and their recommendations for realist RCTs. Bonell et al. are right to question whether and how (quasi-)experimental designs can be improved to better evaluate complex public health interventions. However, the paper does not explain how a research design that is fundamentally built upon a positivist ontological and epistemological position can be meaningfully adapted to allow it to be used from within a realist paradigm. The recommendations for "realist RCTs" do not sufficiently take into account important elements of complexity that pose major challenges for the RCT design. They also ignore key tenets of the realist evaluation approach. We propose that the adjective 'realist' should continue to be used only for studies based on a realist philosophy and whose analytic approach follows the established principles of realist analysis. It seems more correct to call the approach proposed by Bonell and colleagues 'theory informed RCT', which indeed can help in enhancing RCTs.

RAMESES publication standards: meta-narrative reviews.

BACKGROUND: Meta-narrative review is one of an emerging menu of new approaches to qualitative and mixed-method systematic review. A meta-narrative review seeks to illuminate a heterogeneous topic area by highlighting the contrasting and complementary ways in which researchers have studied the same or a similar topic. No previous publication standards exist for the reporting of meta-narrative reviews. This publication standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for meta-narrative reviews. METHODS: We (a) collated and summarized existing literature on the principles of good practice in meta-narrative reviews; (b) considered the extent to which these principles had been followed by published reviews, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, meta-narrative reviews, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing meta-narrative reviews and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence review and real-time problem analysis into a definitive set of standards. RESULTS: We identified nine published meta-narrative reviews, provided real-time support to four ongoing reviews and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature, and common questions and challenges into briefing materials for the Delphi panel, comprising 33 members. Within three rounds this panel had reached consensus on 20 key publication standards, with an overall response rate of 90%. CONCLUSION: This project used multiple sources to draw together evidence and expertise in meta-narrative reviews. For each item we have included an explanation for why it is important and guidance on how it might be reported. Meta-narrative review is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further theoretical and methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of meta-narrative reviews.

RAMESES publication standards: realist syntheses.

BACKGROUND: There is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy-relevant areas - for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for realist systematic reviews. METHODS: We (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing realist syntheses and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real-time problem analysis into a definitive set of standards. RESULTS: We identified 35 published realist syntheses, provided real-time support to 9 on-going syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within three rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%. CONCLUSION: This project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.

RAMESES publication standards: meta-narrative reviews.

BACKGROUND: Meta-narrative review is one of an emerging menu of new approaches to qualitative and mixed-method systematic review. A meta-narrative review seeks to illuminate a heterogeneous topic area by highlighting the contrasting and complementary ways researchers have studied the same or a similar topic. No previous publication standards exist for the reporting of meta-narrative reviews. This publication standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for meta-narrative reviews. DESIGN: A mixed method study synthesising data between 2011 to 2012 from a literature review, online Delphi panel and feedback from training, workshops and email list. METHODS: We: (a) collated and summarized existing literature on the principles of good practice in meta-narrative reviews; (b) considered the extent to which these principles had been followed by published reviews, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, meta-narrative reviews, policy, and/or publishing to produce and iteratively refine a draft set of methodological steps, and publication standards; (d) provided real-time support to ongoing meta-narrative reviews and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence review, and real-time problem analysis into a definitive set of standards. RESULTS: We identified nine published meta-narrative reviews, provided real-time support to four ongoing reviews, and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature, and common questions and challenges into briefing materials for the Delphi panel, comprising 33 members. Within three rounds this panel had reached consensus on 20 key publication standards, with an overall response rate of 90%. CONCLUSIONS: This project used multiple sources to draw together evidence and expertise in meta-narrative reviews. For each item we have included an explanation for why it is important and guidance on how it might be reported. Meta-narrative review is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further theoretical and methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of meta-narrative reviews.

RAMESES publication standards: realist syntheses.

BACKGROUND: There is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy-relevant areas - for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for realist systematic reviews. DESIGN: A mixed method study synthesising data between 2011-2012 from a literature review, online Delphi panel and feedback from training, workshops and email list. METHODS: We: (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigour may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing realist syntheses and the open-access RAMESES online discussion list to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real-time problem analysis into a definitive set of standards. RESULTS: We identified 35 published realist syntheses, provided real-time support to 9 ongoing syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within 3 rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%. CONCLUSIONS: This project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.

Realist methods in medical education research: what are they and what can they contribute?

CONTEXT: Education is a complex intervention which produces different outcomes in different circumstances. Education researchers have long recognised the need to supplement experimental studies of efficacy with a broader range of study designs that will help to unpack the 'how' and 'why' questions and illuminate the many, varied and interdependent mechanisms by which interventions may work (or fail to work) in different contexts. METHODS: One promising approach is realist evaluation, which seeks to establish what works, for whom, in what circumstances, in what respects, to what extent, and why. This paper introduces the realist approach and explains why it is particularly suited to education research. It gives a brief introduction to the philosophical assumptions underlying realist methods and outlines key principles of realist evaluation (designed for empirical studies) and realist review (the application of realist methods to secondary research). DISCUSSION: The paper warns that realist approaches are not a panacea and lists the circumstances in which they are likely to be particularly useful.